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Classic distributed control problems have an interesting dichotomy: they are either trivial or un- 
decidable. If we allow the controllers to fully synchronize, then synthesis is trivial. In this case, 
controllers can effectively act as a single controller with complete information, resulting in a trivial 
control problem. But when we eliminate communication and restrict the supervisors to locally avail- 
able information, the problem becomes undecidable. In this paper we argue in favor of a middle way. 
Communication is, in most applications, expensive, and should hence be minimized. We therefore 
study a solution that tries to communicate only scarcely and, while allowing communication in order 
to make joint decision, favors local decisions over joint decisions that require communication. 

1 Introduction 

Synthesizing code directly from a formal specification is highly intractable. Although automated synthe- 
sis is an attractive concept, neither is the practice of programming currently under threat of extinction, 
nor is automatic synthesis close to become a major factor in code generation. Still, some small critical 
tasks or protocols may be quite tricky for a programmer to produce and can greatly benefit from either 
fully automatic synthesis or a computer assisted development methodology. Prominent representatives 
of such tasks are concurrency control protocols that guarantee mutual exclusion, locking, or efficient 
memory access. The most challenging programming problems are often concurrent in nature, and, alas, 
synthesis of concurrent algorithms is undecidable EOll . 

This undecidability result on synthesizing concurrent code provides an important information about 
how not to attack the synthesis problem: through a general catch-all algorithmic method. One common 
practice to deal with an undecidable result is to restrict the generality of the problem. This can be done by 
limiting the architecture of the system |[20l fT3l IT4l l5l l24l l25ll . Positive results, however, are restricted to 
very limited architectures, such as pipelines, rings, or assumption about the hierarchy of memory access. 

Another approach is to use a heuristic method, accepting that it may not succeed in all cases. A 
genetic search among the space of syntactically limited programs, which mutates existing candidates 
and progresses based on ranking provided by model checking, is described in [7]. Instead of using 
a direct synthesis algorithm, this technique generates candidate solutions, evaluates their quality (the 
model checking is generalized to a fitness function that estimates the distance from a solution), and 
adjusts them to fitter solutions. This method is successful in automatically finding solutions to mutual 
exclusion [7] and leader election problems flU and was even used to detect and correct an error in a 
complicated communication protocol O. In principle, such heuristic search techniques can be fully 
automatic, though they require human interaction, through setting the parameters or adjusting them after 
an unsuccessful run, to be efficient. 

We concentrate on synthesizing distributed control ll22l l23l l29ll . Synthesis is achieved in an incre- 
mental way: an already existing distributed system is modified to satisfy an additional property. In our 
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case, an invariant. Controlling the system is done by selectively blocking transitions. Ideally, local de- 
cisions can be taken by the processes themselves, or equivalently, by supervisors (one per process) that 
control the processes and synchronize with them. It turns out that the controllability problem (whether 
such distributed control exists) is also undecidable ll27l l28l . even for simple safety properties such as 
execution according to priorities [6]. 

To challenge this undecidability result, we relax the problem and allow additional temporary inter- 
actions between processes in order to allow them to acquire sufficient information to decide together on 
allowing (the converse of blocking) a transition. Formally, this coordination is mapped to a supervisor. A 
variant of this method is to partition the processes into groups of communicating processes, or, likewise, 
to introduce regional supervisors and assign each process to one of them. These (regional) supervisors 
collect enough process information to make control decisions. Under this assumption, all processes may, 
at the limit, interact to decide globally on the execution of each transition. This reduces the problem, in 
the limit, to a sequential control problem, which is trivial for finite state systems. The efficiency of this 
method depends on the amount of additional synchronization needed to enforce the desired invariant. 

The method we use to enforce control is based on knowledge JU[T6l. Intuitively, in a distributed 
system, the knowledge of a process includes all properties that globally hold in all states consistent 
with the local view of the process. It reflects limited visibility of processes about the situation in other 
processes. The definition of knowledge is quite subtle, as it involves some assumptions about the view 
of a process. Indeed, in order to make a distributed control decision, a process (or a supervisor process 
synchronized with it) must make a choice that is good for all possible global states that are consistent 
with its local view. As process knowledge may not be sufficient, interaction between processes may 
be used to acquire the joint knowledge of several processes. Furthermore, knowledge can be refined 
based on the history of an execution. In this way, the number of possible global states that are consistent 
with the local view may be reduced, based on different histories. On the other hand, using this kind of 
knowledge requires the support of an expensive program transformation. We will discuss at length the 
use of knowledge in constructing control for distributed systems. 

The knowledge based control synthesis |[T6l [H El restricts the executions of the system. The 
information gathered during the model checking stage is used as a basis for a program transformation that 
controls the execution of the system by adding constraints on the enabledness of transitions. This does 
not produce new program executions or deadlocks and, consequently, preserves all stuttering closed lfl8l 
linear temporal logic properties of the system lfl5l when no fairness is assumed. 

2 Preliminaries 

We chose Petri Nets as our model because of the intuitive and concise representation offered by them. But 
the method and algorithms developed extend to other models, such as transition systems, communicating 
automata, etc. 

Definition 1. A (1-safe) Petri Net N is a tuple (P, T,E,sq) where 

• P is a finite set o/places, 

• the states are defined as S = 2 P where sq £ S is the initial state, 

• T is a finite set of transitions, and 

• E C [P x T) U (r x P) is a bipartite relation between the places and the transitions. 

For a transition t G T, we define the set o/input places *t as {p € P \ (p, t) € E}, and output places t' as 

{ P eP\(t,p)eE}. 
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Figure 1 : A Petri Net 



Definition 2. A transition t is enabled in a state s, denoted s[t), if'tCs and t'Ds C* t. A state s is in 
deadlock if there is no enabled transition from it. 

Definition 3. A transition t can be fired (or executed) from state s to state s', denoted by s[t)s', when t is 
enabled at s. Then, s' = (s \* t) U t*. 

Definition 4. Two transitions t\ and ?2 are dependent if U t\*) n (*t2 U ti) ^ 0. Let D C T x T be the 
dependence relation. Two transitions are independent if they are not dependent. 

Transitions are visualized as lines, places as circles, and the relation E is represented using arrows. 
In Figured] there are places pi, P2, Pi and transitions a,b,c,d. We depict a state by putting full 
circles, called tokens, inside the places of that state. In the example in Figure [Q the initial state so is 
{pi, p2, pi}- The transitions that are enabled from the initial state are a and b. If we fire transition a 
from the initial state, the tokens from p\ and py will be removed, and a token will be placed in p?,. In 
this Petri Net, all transitions are dependent on each other, since they all involve the place pj. Removing 
pi, as in Figure |2j makes both a and c become independent from both b and d. 

Definition 5. An execution of a Petri NetN is a maximal ( i.e., it cannot be extended) alternating sequence 
of states and transitions *o^l J 1^2 J 2 • • •> where sq is the initial state, such that, for each states Sj in the 
sequence, Si[tj + \)si + \. We denote these executions by exec(N). 

For convenience, we sometimes use as executions just the sequence of states, or just the sequence of 
transitions, as will be clear from the context. A state is reachable in a Petri Net if it appears on at least 
one of its executions. We denote the reachable states of a Petri Net A" by reach(N). 

We use places also as state predicates. As usual, we write s \= p t iff p\ G s and extend this in the 
standard way to Boolean combinations on state predicates. For a state s, we denote by (f> v the formula that 
is a conjunction of the places in s and the negated places not in s. Thus, q> s is satisfied exactly by the state 
s. For the Petri Net in Figure [Q the initial state sq satisfies cp i0 = p 1 A P2 A -1/73 A -<p4 A -1/75 A — 1 775 A pj. 
For a set of states Q Q S, let (f>g = V ' sE q^ s , or any logically equivalent propositional formula, be a 
characterizing formula of Q. As usual in logic, when and q)g' characterize sets of states Q and Q', 
respectively, then Q^Q' exactly when cpg — > cpg/. 

An invariant Q of is a subset of the states Q C 2 s ; a net Af satisfies the invariant Q if reach(N) C Q. 
A generalized invariant of Af is a set of pairs / C 5 x T; a net Af satisfies / if, whenever s[t) for a 
reachable s, then (s,t) € /. This covers the above simple case of an invariant by pairing up every state 
that appears in Q with all transitions T. 

Definition 6. An execution of a Petri Net N restricted with respect to a set I C S x T, denoted execj(N), 
is a maximal set of executions so?iSi?2S2 • • • S exec(N) such that, s$ is the initial state, for each states 
Si in the sequence, Si\ti + \)si + {, and furthermore € /. The set of states reachable in execi(N) is 

denoted reachj(N). 
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Figure 2: A Petri Nets with priorities a ^d and b <^c 



Definition 7. For a set of executions X, let pref(X) be the set of prefixes (including full executions) ofX. 

Denote the last state of a finite prefix h of an execution by last(h). 
Lemma 1. reachj(N) C reach(N) and exec i(N) C pref(exec(N)). 

As an example of a property we may want to enforce, consider prioritized executions. 

Definition 8. A Petri Net with priorities is a pair (N , <C) with N a Petri Net and <C a partial order 
relation among the transitions T ofN. 

Let 1^ = {(s,t) | s[t) and Vf' € r^ff') — > t' <C ?}. The set of prioritized executions execj^(N) of 
(./V, <C) is the set of executions restricted to 7<g. The executions of the Petri Net M in Figure |2] (when the 
priorities a <C d and b <C c are no? taken into account) include abed, acbd,bacd, bade, etc. However, the 
prioritized executions of (M, <C) are the same as the executions of the Net Af in Figure Q] 

Definition 9. A process n of a Petri Net N is a subset of the transitions T. 

We will represent the separation of transitions of a Petri Net into processes using dotted lines. We 
assume a given set of processes C that covers all transitions of the net, i.e., \J nec K = T. A transition can 
belong to several processes, e.g., when it models a synchronization between processes. Let procit) = \% \ 
t € 7l} be the set of processes to which t belongs. For the Petri Net in FigureQ] there are two executions: 
acbd and bdac. There are two processes: the left process 71/ = {a,c} and the right process n r = {b,d}. 

The neighborhood of a set of processes IT includes all places that are either inputs or outputs to 
transitions of IT. 

Definition 10. The neighborhood ngb(n) of a process % is the set of places {j ten (*t Lit*). For a set of 
processes IlCc, ngb(Jl) = {J Ken ngb(K). 

A set of processes n owns the places in their neighborhood that can gain or lose a token by a transition 
t only if t is exclusively in IT. 

Definition 11. The set of places owned by a set of processes (including a singleton process) IT, denoted 
own(U), is ngb(U)\ngb(c \ U). 

When a notation refers to a set of processes IT, we will often replace writing the singleton process 
set {tc} by writing 7t, e.g., we write own(n). Note that ngb(Hi) Lingb(Yl2) = ngb(Yli UlL;), while 
own (Hi) Liown(Tl2) C own(Tli UEk). The neighborhood of process 71/ in the Petri Net of Figure [T]is 
{pi,P3,P5,pj}. Place p-] is neither owned by 7t/, nor by 7C r , but it is owned by {7t/,7t r }. It belongs to the 
neighborhood of both processes and acts as a semaphore. It can be captured by the execution of a or of 
b, guaranteeing that — >(/>3 A p$) is an invariant of the system. 

Our goal is to control the system to satisfy a generalized invariant by restricting some of its transitions 
from some of the states. The setting of the control problem may impose that only part of the transitions, 
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ct(T) C T, called controllable transitions, can be selectively supported by the processors that contain it. 
(It blocks if no processor supports it.) The other transitions, uc(T) = T\ct(T), are uncontrollable. Note 
that we may be at some state where either some uncontrollable transitions, or all enabled transitions, 
violate the generalized invariant. Being in such states is therefore "too late"; part of the controlling task 
is to avoid reaching such states. 

In control theory, the transformation that takes a system and allows blocking some transitions adds 
a supervisor process [21], which is usually an automaton that runs synchronously with the controlled 
system. This (finite state) automaton observes the controlled system, progresses according to the transi- 
tions it observes, and blocks some of the enabled transitions, depending on its current state. In a similar 
way, in distributed control ||29l l23l 1221 . for each process we assign such a supervisor, which changes 
its states each time the process it supervises makes a transition, or when a visible transition of another 
process (e.g., through the change of shared variables) is executed. Based on its states, the supervisor 
allows (supports) transitions of the controlled process. In a disjunctive control architecture [29], if no 
supervisor suports an, otherwise enabled, transition, it cannot execute and is thus blocked. Such a super- 
visor can be amalgamated, through a transformation, into the code of the controlled process. In order to 
capture this for Petri Nets, without a complicated transition splitting transformation, we use an extended 
model, as defined below. In particular, it allows adding enabling conditions and variable transformation 
to capture the encoding of the local supervision of the processes. It would also allow encoding additional 
asynchronous supervision in our solution. 

Definition 12. An extended Petri Net / [72l/ is a Petri Net with a finite set of variables V K over a finite 
domain per each process 71 G IT. In addition, a transition t can be augmented with a predicate en t on 
the variables V t = ^%epmc(t)^n an d a transformation function f(V t ). In order for t to fire, en t must 
hold in addition to the basic Petri Net enabling condition on the input and output places of t. When 
t fires, in addition to the usual changes to the tokens, the variables V t are updated according to the 
transformation f. 

We transform a Petri Net N and a generalized invariant / into an extended Petri Net N' that allows 
only the executions of N controlled to satisfy /. 

Definition 13. A controlling transformation obeys the following conditions: 

• New transitions and places can be added. 

• The input and output places of the new transitions are disjoint from the existing places. 

• Variables, conditions and transformations can be added to existing transitions. 

• Existing transitions will remain with the same input and output places. 

• It is not possible to fire from some point an infinite sequence consisting of only added transitions. 

Added transitions are grouped into new (supervisory) processes. Added variables will represent some 
knowledge-dependent finite memory for controlling the system, and some interprocess communication 
media between the original processes and the added ones. Processes from the original net will have dis- 
joint sets of variables from one another. The independence between the original transitions is preserved 
by the transformation, although some coordination may be enforced indirectly through the interaction 
with the new supervisory processes. 

Definition 14. Let s\ c map a state s of the transformed version N' into the places of the original version 
N by projecting out additional variables and places that N' may have on top of the places of N. This 
definition is also extended to executions (as sequences of states). 
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This projection allows us to relate the sets of states of the original and transformed version. Firing 
of a transitions added by the controlling transformation does not change s \ c and is not considered to 
violate I (the requirement that (sj,ti + i) in Definition [6]is imposed only when ti + \ is from the original net 
AO. Note that our restrictions on the transformation implies that the sets ngb(H) and ownill) for IT C c 
are not affected by the transformation. Furthermore, albeit the rich structure of extended Petri Nets, our 
control transformation will allow a finite state control for a finite state system. 

Definition 15. Two executions G and o', viewed as sequences of states, are equivalent up to stutter- 
ing H18\l when, by replacing any finite adjacent repetition of the same state by a single occurrence in 
both G or o', we obtain the same sequence. Let stutcliT) be the stuttering closure of a set T of sequences, 
i.e., all sequences that are stuttering equivalent to some sequences in T. 

Lemma 2. A controlling transformation produces an extended Petri Net N' from N such that 
exec(N') \ c C pref \stutcl(exec(N))). 

The controlling transformation may introduce new deadlocks, hence the lemma above asserts about 
the prefixes of the original executions. Of course, this is not a desirable outcome of the control transfor- 
mation, and the solutions that will be given to the distributed control problem will circumvent it. 

3 Process Knowledge and Joint Process Knowledge 

The knowledge of a process at a given execution point consists of facts that hold in all global states 
that are consistent with the current local view of this process. The current local view represents the 
limited ability of a process to observe the global state of the system. A process may be aware of its own 
local variables and shared variables in its neighborhood. Similarly, we can define the joint knowledge of 
several processes, by considering their joint local view. 

According to the limited observability of the processes n, we can define an equivalence relation 
EnC S x S (when the set of processes II is a singleton, we can write = K ) among the states S of the 
system; if the current state is s € S, then the processes IT cannot distinguish, given their joint local view, 
between s and any state equivalent to it according to =ri- Such an equivalence relation is the basis of the 
definition of knowledge (H. 

Definition 16. The processes IT (jointly) know a property \|/ in a state s, denoted s (= ^n^» if far all s' 
such that s = n s', we have that s' (= \\f. 

In the Petri Nets model, the equivalence relation = n can be defined by restricting first each state 
to a part of a state. Then, states that share the same part are considered equivalent. There are several 
possibilities to restrict the part of a state that is associated with a subset of the processes II. We will give 
two possibilities for such a restriction. The first one is that of local information, which takes the part 
of the state that includes the neighborhood of the processes IT. This Petri Nets definition corresponds, 
in general systems, to the variables that can be read or written by the processes IT. The second such 
restriction is that of local state (different names were chosen only to make a distinction), based on 
restricting states to the places that the processes IT own. This corresponds, in general systems, to the 
variables that only the processes IT, and no other processes, can change (write). 

Definition 17. The local information of a set of processes IT of a Petri Net N in a state s is s\n= 
sDnbg(U). 

In the Petri Net in Figure [Q the local information of 71/ in any state s consists of the restriction of s to 
the places {pi,P3,P5,pi}. In the depicted initial state, the local information is {p\,pj}. 
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Definition 18. The local state of a set of processes IT of a Petri Net N in a state s is s\jj= s H own(H). 

It is always the case that s[nQ s \n- The local state of 71/ in the initial state of Figure [TJ is {p\}. 
Lemma 3. IfK £ IT then sLnu{7t} w ^ g (disjoint) union of s |_n a«J 5[jtnow«(nu {7t}). 

In the following definitions, we can often use either the local information or the local state. When 
this is the case, we will use s\n instead of either s\jj or *[n- 

Definition 19. Let Tl Q C be a set of processes. Define an equivalence relation =n^ reach(N) x 
reach(N) such that s =n when s\n = s'|n- 

As s\n can stand for either s\u or s|_ri> this gives two different equivalence relations. When it is 
important to distinguish between them, we denote the one based on "[" as =J^ (weak equivalence) and 
the one based on "|_" as =fj (strong equivalence). 
Lemma 4. Ift £ n and s s' then s[t) if and only if s'[t). 

That is, the enabledness of a transition depends only on the local information of a process that 
contains it. This does not hold when we replace by =£.. In the Prioritized Petri Net in Figure [TJ 
e.g., we have that {pi,P2,Pi} {pi^Pa,Pi}, since 71/ has the same local information {p\,pi} in both 
states. The state {p\,pa} is not equivalent to either of these states. On the other hand, these three states 
are equivalent according to = s % (pi is not in own(%i)). 

Corresponding with the two equivalence relations of Definition [T9l we distinguish between knowl- 
edge based on strong equivalence =jj (and hence on local states), denoted and knowledge based 
on weak equivalence =J^ (and hence local information), denoted K^y. The knowledge based on the 
local state (resp. local information) is called strong (resp. weak) knowledge. Since the local information 
determines the local state (while multiple local states may have the same local information), we have 
Kfj(p — > A^cp. Consequently, we may know more under weak knowledge. 

The motivation for the different definitions of equivalence and, subsequently, the different defini- 
tions of knowledge is as follows. In order to make choices (to support or block a transition) that take into 
account knowledge based on local information, a process, or a set of processes, needs to have some guar- 
antee that the local information will not be changed by other processes while it is collecting information 
from the processes or making the decision. For a single process, this may be achieved by the underlying 
hardware. But it is unreasonable to require such a guarantee for a set of processes that either temporary 
interact (interactions take time and other processes may meanwhile progress) or send their current local 
view to some supervisor process that collects views from several processes. Thus, for decisions involving 
a set of processes, strong knowledge, based on the joint local state, is used instead. 

The classical definition of knowledge is based on relations =n over the reachable states reachj(N). 
However, when using knowledge to control a system to satisfy a generalized invariant, one may calculate 
the equivalences and the knowledge based on the states reachj(N) that appear in the executions of the 
original system that satisfy this generalized invariant /. This (cyclic looking) claim is proved Q by 
induction on the progress of the execution in the controlled system: for a state already on such an 
execution (by the inductive assumption) the controlled system allows firing only transitions that preserve 
the generalized invariant, hence is also in reach j (N). We may need to restrict the generalized invariant /, 
in order not to introduce new deadlocks. This means even fewer reachable states, which can consequently 
increase the knowledge further. 

One of the main challenges of using knowledge for controlling systems is that it is not always possible 
to decide, based on the local (or joint) knowledge, whether or not allowing a transition will guarantee 
the desired generalized invariant. One tool that can be used in this case is to allow additional interactions 
between processes, or knowledge accumulation by additional asynchronous supervisors. This will be 
explained later. However, before progressing to such an expensive solution, we may also try to improve 



Down Peled and Sven Schewe 



9 



the knowledge by refining the equivalence relation that is used in its definition. 

The definitions of knowledge that we used assumes that the processes do not maintain a log with their 
history. The use of knowledge with such a log, called knowledge with perfect recall lfl6l . is discussed 
in HI. Consider an equivalence & n between histories that seem undistinguishable to the process 71. Two 
finite prefixes h, h! of Petri Net executions will be considered equivalent for r% if the projection of h 
on transitions visible to ji are the same in both h and h' . Specifically for Petri Nets, we can define the 
transitions vis(%) = {t\('tL)t') r\ngb(n) ^ 0} (t is dependent on some transitions in Ji). In this case, the 
last states last(h) and last(h') of h and h', respectively, are equivalent under = w (and hence also under 
= s ). This can be shown by induction over the length of the prefixes, based on the fact that only the 
transitions in vis(n) affect ngb(%) D own(%). 

Definition 20. Let h\=^f exactly when last(h) \= \|/. Then we define past knowledge, where h \= K%y\r if, 
for all h' ~ n h, h\= \|/. 

In particular for properties \|/ that depend only on the last state of h, the use of the history refines 
the weak equivalence between states: h r% h' implies last{h) =^ last(h'). To take advantage of the 
refined definition of knowledge, we need somehow to distinguish local states that have non equivalent 
histories. On the face of it, this seems to require unbounded memory. However, looking deeper into the 
new definition of knowledge, one can observe that the following finite construction will work |[T6l lTi. 

Definition 21. Let A K be the set of finite sequences of transitions that do not change the neighborhood 
of '71 (i.e., independent with the transitions in Tl). 

Definition 22. Let SI = (S, so, T) be a finite automaton representing the global states Sofa Petri Net N, 
including the initial state sq E 5 and the transitions T between them. For each process 7C, we construct an 
automaton Sl n representing the set of states of SI where the Petri Net N can be after a given local history. 
The automaton SL n has the following components: 

• The set of states is 2 s . 

• The initial state is the set of states {s\3u £ A K s.t.so[u)s}. That is, the initial state of this automaton 
contains all states obtained from so by executing a finite number of transitions independent of (i.e., 
invisible to ) 7t. 

• The transition relation is F — V between two states T, T' £ 2 s and a transition t is as follows: 
T' = {s'\3s € TBp. € A K s.t.,s[t[i)s'}. That is, a move from F to F' corresponds to the execution of 
a transition t that changes the neighborhood of K followed by transitions independent ofK. 

Then, one may use instead of for locally supporting transitions. (Note that — > K%.) 
However, the size of each such automaton (one per process 7t) can be exponential in the size of the global 
state space. Knowledge of perfect recall can be implemented by adding a synchronized supervisor with 
memory (basically implementing the automaton SA K ). It is natural to ask whether one can make an even 
finer distinction between states than with knowledge of perfect recall. This is indeed possible, but at the 
cost of a more involved program transformation. We may augment in our transformation the context of 
the interprocess communication between processes with additional transformation, that would implement 
the support for additional knowledge. Such a transformation can, e.g., be based on Gossip Automata ifTTIl . 
providing the most recent past local view of any other process. 

We henceforth use knowledge formulas combined with Boolean operators and propositions. For a 
detailed syntactic and semantic description of logics with knowledge one can refer, e.g., to H]. Once s \= 
Kny is defined, \|/ can also be a knowledge property, hence s \= K u /Ku^ (knowledge about knowledge) is 
also defined, though the finite-state representation described above only applies to past knowledge used 
in outermost knowledge operators. 
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Lemma 5. If s \= ^ n cp and s = n s', then s' (= K n q>. 

Lemma 6. Knowledge is monotonic with respect to the set of observing processes: if IT C IT then 

Lemma 7. Given that s \= £ncp in some basic Petri Net N, then s \= Ku^> also in a transformed version N'. 

Enforcing prioritized executions in a completely distributed way may be impossible. In Figure [2 a 
and c belong to the left process 71/ , and b and d belong to the right process n r , with no interaction between 
the processes. The left process 71/, upon having a token in p\, cannot locally decide whether to execute 
a; the priorities dictate that a can be executed if d is not enabled, since a has a lower priority than d. But 
cannot distinguish between the cases where 7t r has a token in P2, Pa, or p(,. 

In the Prioritized Petri Net in Figure|2j e.g., we have that {p\,P2} =%, {p\,Pa}, since in both states 
7t/ has the same local information {pi}. In the state {pi,P2}, a is a maximal priority enabled transition 
(incomparable with b), while in {p\,Pa\, a is not maximal anymore, as we have that a -C d, and both 
a and d are now enabled. In the initial state the local information (and also the local state) of 7t/ is 
{p\}. Thus, 7t/ does not have enough knowledge to support any transition since {^1,^2} {pi,P3})- 
Similarly, the local information of 7t r is {P2}, which also is not sufficient to support any transition. After 
they both hang on a supervisor, it has enough information to support a or b. 

4 A Globally Controlled System 

Before providing a solution to the distributed control problem we need to provide a solution to the related 
global control problem. Some reachable states are not allowed according to the generalized invariant. 
In order not to reach these states, resulting in an immediately deadlock, we may need to avoid some 
transitions that lead to such states from previous states. This is done using game theoretical search. 

The game is played between a constructor, who wants to preserve the generalized invariant / indef- 
initely (or reach a state that is already a deadlock in the original system AO, and a spoiler, who has the 
opposite goal. The game is played on the states S of a net. It starts from the initial state so and ends 
if a deadlock state is reached (and may go on forever). In each round, the constructor player chooses a 
nonempty subset of enabled transitions that must include all enabled uncontrollable transitions. Subse- 
quently, the spoiler chooses a transition from this set, which is then executed. The spoiler wins as soon 
as she can choose a transition that violates /, i.e., (s,t) £ I, while the constructor wins if this condition 
never holds (on an infinite run or a finite run that ends in a deadlock). 

We can define an "attractor" attr(A) that contains all states in A and all states that the spoiler can 
force to A in a single transition. A state s is in attr{A) if one of the following conditions holds: 

• sGA, 

• there exists an uncontrollable transition t € uc(T) enabled in s with s[t)s' and either s' € A, or 

(s,t) or 

• s is not a deadlock state in the Petri Net N and, for all transitions t enabled in s, such that s[t)s' and 
(s,t) G /, it holds that s' € A. 

As usual, we define attr n+l (A) = attr(attr n (A)), where attr°(A) = A. Because of the monotonicity 
of the attr(A) operator (with respect to set inclusion) and the finiteness of the state space, there is a least 
fixpoint attr*(A), which is attr n (A) = attr n+l {A) for some (smallest) n. 

Now, let I G = {(s,t) G / | s[t)s' and s' attr*(%)}. Let G = reachj G (N) if s attr*{%), otherwise 
G = 0. These are the "good" reachable states in the sense that they are allowed by / and the system can 
be controlled to henceforth adhere to /. 
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Definition 23. Let R = {(s,t) £ I \ 3s' s[t)s' As,s' £ G} be the safe transition relation. 

If the initial state is good (sq £ G), then the constructor can win by playing according to 7?. If, on 
the other hand, sq is in the attractor attr*{%) of the bad states, then sq is in attr n {%) for some n < \S\. By 
the definition of attr n ($), the spoiler can force the game to attr n ~ l ($) in the next step, then to attr n ~ 2 {$), 
and so forth, and thus make sure the bad states are reached within at most n steps. 

Lemma 8. The constructor can force a win if and only if sq £ G. 

This game can obviously be evaluated quickly on the explicit game graph, and hence in time expo- 
nentially in the number of places. EXPTIME completeness can be demonstrated by a simple reduction 
from the PEEK-G5 [26] game iflOl . Deciding if the constructor can force a win is PSPACE complete for 
Petri Nets with only controllable transitions [10]. 

Model Checking 

We will use the following propositional formulas, with propositions that are the places of the Petri Net: 

- The good states G: (p G - 

- The states where a transition t is enabled: cp e „( ( ). 

- At least one transition is enabled, i.e., there is no deadlock: cp^/ = \lteT tyen(t)- 

- Transition t is allowed from the current state by the safe transition relation R: ty g00 d{t) 

- The local information (resp. local state) of processes IT at state s: (p s r n (resp. cp^n)- 

The corresponding sets of states can easily be computed by model checking and stored in a compact 
way, e.g., using BDDs. Given a Petri Net, one can perform model checking in order to calculate whether 
s \= K % y\r. The processes IT know \|/ at state s exactly when (q> G A <p s i n ) — )• \|/ is a propositional tautology. 
We can also check properties that include nested knowledge by simply checking first the innermost 
knowledge properties and marking the states with additional propositions for these innermost properties. 

Model checking knowledge using BDDs is not the most space efficient way of checking knowledge 
properties, since cp G can be exponentially big in the size of the Petri Net. In a (polynomial) space 
efficient check (which has a higher time complexity), we enumerate all states s' such that s = % s' , check 
reachability of s' using binary search, and, if reachable, check whether s' \= \|/. This can also be applied 
to nested knowledge formulas, where inner knowledge properties are recursively reevaluated each time 
they are needed. The PSPACE complexity is subsumed by the EXPTIME complexity in the general case 
algorithm for the safe transition relation R. 

5 Control Using Knowledge Accumulation 

According to the knowledge based approach to distributed control IU|6l|2l[22l, model checking of know- 
ledge properties is used at a preliminary stage to determine when, depending the local information, an 
enabled transition can safely be fired. In our case, this means checking s \= K^^> good ^ (by Lemma[5l the 
satisfaction only depends on s\ K ). At runtime, a process supports a transition in every local information 
where this holds. The following support policy uses this information at runtime: 

A transition t can be fired (is enabled) in a state when, in addition to its original enabledness 
condition, at least one of the processes in proc(t) supports it. 

Enabled uncontrolled transitions can always be supported, as a consequence of the following Lemma. 
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Lemma 9. Ift e ft[~lwc(r) and (s,t) G /?, f/i<?« 5 |= K™<p good t t y 

This follows from the observation that the safe transition relation does not restrict the uncontrolled 
transition. 

It is possible that, in some (non deadlock) states of G, no process has enough local knowledge to 
support an enabled transition and, furthermore, no uncontrollable transitions are enabled. We may need 
to synchronize several processes or collect the joint knowledge of several processes through the use of 
asynchronous supervisors. A process can decide, based on its current (lack of) knowledge, whether 
it hangs on such supervisor by sending it its local state. A supervisor 1 can make a decision, based 
on accumulated joined knowledge of several hung processes, that one of them can support an enabled 
transition. A process hangs on a supervisor, when the following property does not hold: 

K 7C = \jKgy ggod(t) VK% V V^>WW 
ten K'yLn ten' 

That is, a process does neither hang on the supervisor when it has enough knowledge to support a tran- 
sition, nor if it knows that some other process has such knowledge. In the latter case, it does not actually 
need to be able to determine which process has that knowledge. 

To avoid the overhead of computing past knowledge, it is often cheaper (and more appropriate) to 
use weak knowledge instead. In case nested knowledge calculation is too expensive as well, we may 
use the simplified knowledge formula \/ te% K£ '<p g00 d(t ) instead, at the expense of making more processes 
hang. 

The supervisor T keeps the updated joint local state of the hung processes IT. When a process 7t 
hangs, it updates this view by transmitting to 1 its local information s\ % , from which T keeps (according 
to Lemma© s\ n C\own(T\\J {ft}). Since all processes in IT = IlU {ft} are now hung, no other process can 
change these places. Then the joint knowledge K^,(^ goo M t \ can be used to support a transition t. Recall 
that knowledge based decisions of a single process use weak knowledge (based on the local information), 
while multiple processes use strong knowledge (i.e., based on the joint local state). 

In the following cases, 

1. after the decision of a process ft to hang on T , other processes make changes to ft's local informa- 
tion that allow it to support some transition t, 

2. when a transition t with {ft, ft'} C proc{t) is supported by ft' while ft is hung, or 

3. when an uncontrollable transition executed (which is enabled even if it belongs to a hung process), 

we allow ft to notify 1 that it has decided not to hang on it anymore. Moreover, T , which acquired 
information about the hung processes IT, will have to forget the information about the places own(H) \ 
own(Jl \ {ft}). The ability of processes to hang on a supervisor but also to progress independently before 
the supervisor has made any supporting choice requires some protocol between the processes and the 
supervisor. 

Instead of having a single supervisor T, we can use several supervisors T\, I2, ■ . ■ , Tk, where each 
supervisor % takes care of a set of processes proc(Tj). These sets are pairwise disjoint and do not 
necessarily cover all processes. 

An effectively checkable criterion to determinte if at least one process or supervisor will be able to 
provide a progress from any nondeadlock state in G is as follows: 

(cp G A(p rf/ )^ ( \/ K%ood{t)V V V Kroci^goodit)) 

tenec iel...k teTieproc{ij) 
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Lemma 10. Under our transformation from a Petri Net N to an extended Petri Net N', exec(N') \ C C. 
stutcl(exec[(N)) holds. 

This is proved by induction on prefixes of the execution and using Lemma|2] 
Lemma 11. N' satisfies all stuttering invariant temporal properties ofN. 

Implementing the Supervisors 

Processes hang on a supervisor in some arbitrary order. The supervisor needs to decide, based on the part 
of the global state that it sees, whether or not there is enough information to support some transition. 
Definition 24. Let L = {s[nxTl\ s £ G,H C c} denote the set of joint local states, each paired up with 
the set of relevant processes (then G x C Q L). We define CCLx L (and, symmetrically, □) as follows: 
q^q' if q = (s[m ,^li),q' = (sLrhil^) (i.e., both are part of the same global state s) and Hi C Yl 2 . We 
say that q' subsumes q. 

Definition 25. The support function supp : L — > 2 T returns, for each q € L, the transitions that are 
allowed by Rfrom all states that subsume q. Formally, supp(q) = H( s c p^{? | t £ T, (s,t) € R}. 

That is, for q = (j|_n>n), t € supp(q) iff s \= K^q> good ( t y If t € supp(q)C\ct(T), then the supervisor 
can select a process in proc(t) to support t. Obviously, when q C q' , supp(q) C supp(q'). There is no need 
for a supervisor to store in the domain of supp elements q = (s|_n,n) where | O | < 2: when supp{q) 7^ 0, 
the process with this local state can locally support a transition without the help of a supervisor. 
Definition 26. Let L x L be such that q ~> q' if q = (s\j\,T\) and q' = (sLnu{7i}>n U {k}), where 
71 n (i.e., q' extends q according to exactly one process). 

The supervisor updates its view about the joint local state of the processes according to the relation 
~K when moving from q to q' by acquiring the relevant information about a new processor 71; conse- 
quently, its knowledge grows and it can decide to support one of the transitions in supp(q'). 
Definition 27. A joint local state q is minimal supporting if supp (q) ^ and, for each q' such that q' q, 
supp(q') = 0. 

Definition 28. The upward closure of a subset of the joint local states U QLis {q €L \ 3q' £Uq' C 
q}. 

Lemma 12. A sufficient condition for restricting the domain U C L of supp for a supervisor, without 
introducing new deadlocks, is that Gx{c) C'f [/. 

Thus, there is no need to calculate and store all the cases of the function supp. This suggests the 
following algorithm for calculating the representation table for supp: perform DFS such that if q^ q', 
then q is searched before q'; backtrack when visiting q again, or when supp(q) ^ 0. This algorithm can 
be used also for multiple supervisors, when restricting the search to the joint local states of IT C proc(Ti) 
for each T,-. 

In order to reduce the set of local states that a supervisor needs to keep in the support table, one 
may decide that a supervisor will not always support transitions as soon as the joint local state of the 
hung processes allows that. This introduces further delays in decisions, where the supervisor waits for 
more processes to hang even when it can already support some transitions. On the other hand, the set of 
supported transitions may be larger in this case, allowing more nondeterminism. 

The size of the global state space of a Petri Net is in 0(2^). Since we need to keep also the joint 
local states, the size of the support table that we store in a supervisor, is in o(2l p l + l c l) (which is the size 
of L). However, by Lemma [T2l the representation may be much more succinct. In theory, when there 
are no uncontrollable transitions, a (particularly slow) supervisor can avoid storing the support table, and 
perform the PSPACE binary search each time it needs to make a decision on a joint local state. 
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Control Through Temporary Interaction 

The control solution suggested here makes use of (semi-)global supervisor(s) to accumulate the joint 
local states of several processes, when these processes cannot locally support transitions based on their 
weak (or past) knowledge. In |6j, a solution based on temporary synchronization between the processes 
was suggested. Preference is given to supporting transitions locally. However, when the local knowledge 
is not enough to support a transition based on the local information (including the case where it is 
known that some other process currently has the knowledge), i.e., K 71 does not hold, the process tries 
to synchronize with other processes in order to achieve joint knowledge. 

In order to put the solution in [6] in the context of the construction here, each process is, upon 
reaching a state with local information where K n does not hold, willing to be involved in interactions 
according to U. In order to implement this, each process maintains, for each local state (or, when using 
past knowledge, for each history), the set of joint local states that contain its local state, and where supp 
supports at least one transition x. Upon reaching that local state, the process is willing to participate in 
interactions consisting of such joint local states. A successful interaction will allow firing transitions 
according to supp. 

The coordination is facilitated through a protocol such as the a-core. The a-core protocol, as de- 
scribed in [19] contains a small error, which was automatically corrected using a genetic programming 
tool in [9]. Each interaction consists of exchanging of some messages, to request interaction, to allow it, 
to confirm the interaction or to cancel it, etc. Obviously, there is quite a lot of overhead involved. 

There are advantages and disadvantages to both approaches: using a (semi-)global supervisor and 
using temporary syncrhonization. In particular, the latter is more flexible, as several interactions may be 
performed in parallel, and there is no need to commit on the distribution of processes to the semiglobal 
supervisors. On the other hand, it seems to require more overhead. 

6 Reducing Process Hanging and Passing Responsibility 

The introduction of a partial order y on the set of processes leads to a situation, where a smaller process 
w.r.t. y can avoid hanging on its supervisor if the bigger processes together can progress. Besides the 
advantage of reducing the number of calls to supervisors, it also allows for providing a preference to 
important processes, giving them an advanced access to supervisor support while reducing supervisor 
interaction for lesser processes significantly. 

This makes use of nested knowledge, a generalization of the property K 71 to a set of processes 
kFI Vreun KfiWgood(t) ■ 

The intuition is that a process can check whether it knows that the joint knowledge of the other 
processes, besides itself, is sufficient to support a transition, i.e., K^K c \^ n \ In this case, a process 
may decide not to hang, but to rather let the others provide the joint local state needed for making 
the progress decision. However, this solution makes it possible that too many processes will decide to 
delegate responsibility to others, without informing them. This can lead to the introduction of a deadlock. 

The use of the partial order >- circumvents this problem. For a supervisor % we use n, = proc(Ti) 
to denote the processes it supervises. For a process 71, we denote with Yl^ n = {%' G IT,- [ Tt' >- 7C} the 
processes of II,- that are strictly greater than 7C with respect to the partial order K Naturally, a supervisor 
% would support some transition based on the knowledge of the processes in IT^ 71 if K n < holds. A 
process 71 can thus idle if it knows Kg Vn-e^ ^ ■ This * s usec ^ to reduce the states in which a process 
hangs on its supervisor. 

The control strategy of the supervisors is not affected. The ordered control strategy is as follows: 
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1. If a process % knows that a transition is good, then it supports it. 

2. Otherwise, if a process ji knows that, for some transition t £ 71, a different process knows that t is 
good, then n idles. 

3. Otherwise, if a process 7t knows that, for some supervisor Tj, the joint knowledge of H^ n is that 
some t G Yl^ n is good, then 7C idles. 

4. Otherwise, ji hangs on its supervisor. 
Ordered control does not introduce new deadlocks. 

7 Conclusions 

We presented simple and effective algorithms for synthesizing distributed control. The resulting control 
strategy uses communication and knowledge collection without blocking the processes unnecessarily. 
One strength of our approach is that it is complete in the sense that, provided a centralized solution 
exists, it finds a solution. However, this does not come at the cost of centralizing the control completely. 
To the contrary, the system can progress without the support of a global or regional supervisor as soon 
as the local information suffices to do so. 

Our solution for the distributed control of systems uses knowledge to construct a distributed con- 
troller for a global constraint. In flT] |2j, it is demonstrated that the local knowledge may be insufficient to 
construct a controller. Knowledge of perfect recall lfl6l . which depends not only on the local state (infor- 
mation), but on the gathered visible history, can alleviate some, but not all, of these situations. The use of 
interprocess communication to obtain joint knowledge is suggested in 11221 : however, no systematic algo- 
rithm for collecting such knowledge, or for evaluating when enough knowledge has been collected, was 
provided there. In [6], joint knowledge is calculated through temporary multiprocess synchronization. 
However, such synchronization is expensive, and multiple interactions (including different interactions 
of the same set of processes) may require a separate synchronizing process. We presented here a practi- 
cal solution, based on 0] 12 |6l [lOl [HI for distribute control where a small number of (or even a single) 
supervisor(s) run(s) concurrently with the controlled system. 

While the classical synthesis problems for concurrent control of distributed systems are undecidabil- 
ity |[20ll24ll27l [281. we relax the synthesis assumption to allow additional interactions, when needed. We 
believe that this makes a practical basis for synthesizing control for distributed systems. These methods 
were implemented [TOl [TTJ . There are various tradeoffs in the approaches presented, which calls for 
further experiments and tuning. 
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